Developments in the fields of robotic prosthesis have made it possible to build devices that mimic a human in its ability to manipulate multiple degrees of freedom simultaneously (e.g., the modular prosthetic limb developed by the Johns Hopkins University Army Physical Laboratories). However, designing strategies to control such prostheses, especially from neural activity, remains a challenge. Such a synergy between the hardware and its control using thought poses immense benefits for medical, rehabilitation and motor performance enhancements.
The field of brain computer interface (BCI) systems or brain machine interface (BMI) systems deals with interpreting the neural code and generating commands to control an assistive device. The terms brain computer interface (BCI) and brain machine interface (BMI) are used interchangeably herein. BCI systems may thus potentially provide movement-impaired persons with the ability to interact with their environment using only their thoughts to control assistive devices such as communication programs and smart artificial arms. Some BCI systems rely on neural signals acquired noninvasively with electroencephalography (EEG) (Wolpaw et al. (2004) “Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans,” Proc. Natl. Acad. Sci. USA, 101:17849-17854), while other BCI systems acquire signals invasively with electrocorticography (ECoG) (Schalk et al. (2008) “Two-dimensional movement control using electrocorticographic signals in humans,” J. Neural Eng., 5:75-84) or microelectrode arrays seated into cortical tissue (Hochberg et al (2006) “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, 442:164-171).
Invasive BCI systems typically acquire neural signals with intracranial or subdural electrodes, while noninvasive BCI systems typically acquire neural signals with scalp EEG. Some drawbacks of invasive BCI systems are the inherent risks of surgery and gradual degradation of signal integrity. However, a limitation of conventional noninvasive BCI systems for two-dimensional control of a cursor, in particular those based on sensorimotor rhythms, is the lengthy training time required by users to achieve satisfactory performance.
With regard to invasive systems, researchers have extracted hand trajectories or velocity profiles from neuronal signals acquired with electrodes seated directly into cortical tissue and, in some cases, used these kinematics to command a robotic arm in real time (Wessberg et al. (2000) “Real-time prediction of hand trajectory by ensembles of cortical neurons in primates,” Nature, 404:361-365; Serruya et al. (2002) “Instant neural control of a movement signal,” Nature, 416:141-142; Taylor et al. (2002) “Direct cortical control of 3D neuroprosthetic devices,” Science, 296:1829-1832; Hochberg et al (2006), supra, Nature, 442:164-171; Kim et al. (2006) “A comparison of optimal MIMO linear and nonlinear models for brain-machine interfaces,” J. Neural Eng., 3:145-161; Mulliken et al. (2008) “Decoding trajectories from posterior parietal cortex ensembles,” J. Neurosci., 28:12913-12926; Truccolo et al. (2008) “Primary motor cortex tuning to intended movement kinematics in humans with tetraplegia,” J. Neurosci., 28:1163-1178; Velliste et al. (2008) “Cortical control of a prosthetic arm for self-feeding,” Nature, 453:1098-1101). Investigators have also extracted hand kinematics from intracranial local field potentials obtained through less invasive ECoG (Schalk et al. (2007) “Decoding two-dimensional movement trajectories using electrocorticographic signals in humans,” J. Neural. Eng., 4:264-275; Pistohl et al. (2008) “Prediction of arm movement trajectories from ECoG-recordings in humans,” J. Neurosci. Methods, 167:105-114; Sanchez et al. (2008) “Extraction and localization of mesoscopic motor control signals for human ECoG neuroprosthetics,” J. Neurosci. Methods, 167:63-81).
However, little work has been done to continuously decode natural, multi-joint limb kinematics from neural signals acquired noninvasively. Only a few studies report continuous decoding of two-dimensional (2D) hand and tool kinematics from magnetoencephalography (MEG) (Georgopoulos et al. (2005) “Magnetoencephalographic signals predict movement trajectory in space,” Exp. Brain Res., 167:132-135; Jerbi et al. (2007) “Coherent neural representation of hand speed in humans revealed by MEG imaging,” Proc. Natl. Acad. Sci. USA, 104:7676-7681; Bradberry et al (2008) “Decoding hand and cursor kinematics from magnetoencephalographic signals during tool use,” Conf. Proc. Eng. Med. Biol. Soc. 2008:5306-5309; Bradberry et al. (2009a) “Decoding center-out hand velocity from MEG signals during visuomotor adaptation,” Neuro-image, 47:1691-1700). However, such MEG systems are immobile and therefore unsuitable for practical BCI systems.
Researchers have not demonstrated continuous decoding of limb kinematics from EEG-based systems. Instead, most EEG studies have discretely classified the direction/speed of 2D hand/wrist movements or different motor imagery tasks on a single-trial basis (Mellinger et al. (2007) “An MEG-based brain-computer interface (BCI),” Neuroimage, 36:581-593; Hammon et al. (2008) “Predicting reaching targets from human EEG,” IEEE Signal Proc. Mag., 25:69-77; Walden et al. (2008) “Hand movement direction decoded from MEG and EEG,” J. Neurosci., 28:1000-1008; Gu et al. (2009) “Offline identification of imagined speed of wrist movements in paralyzed ALS patients from single-trial EEG,” Front Neuroprosth. 1:1-7) or they have demonstrated 2D continuous control of a cursor through biofeedback training (Wolpaw et al. (2004), supra, Proc. Natl. Acad. Sci. USA, 101:17849-17854). The lack of attention to reconstructing kinematics of natural limb (e.g. hand) movements from EEG is due in part because researchers consider training subjects to modulate EEG activity, independent of reconstructing hand kinematics, suffices for 2D control (Wolpaw et al. (2004), supra, Proc. Natl. Acad. Sci. USA, 101:17849-17854). Further, it is generally thought that the signal-to-noise ratio, the bandwidth, and the information content of neural data acquired via noninvasive scalp EEG are insufficient to extract sufficiently detailed information about natural, multi-joint movements of the upper limb (Lebedev et al. (2006) “Brain-machine interfaces: past, present and future,” Trends Neurosci., 29:536-546).
Current noninvasive EEG-based BCI systems for 2D cursor control require subjects to learn how to modulate specific frequency bands of neural activity, i.e. sensorimotor rhythms, to move a cursor to acquire targets (Wolpaw et al. (2004), supra, Proc. Natl. Acad. Sci. USA, 101:17849-17854). These types of studies based on sensorimotor rhythms require weeks to months of training before satisfactory levels of performance are attained. Relative to EEG signals, the increased signal-to-noise ratio and bandwidth of invasively acquired neural data are commonly thought to be factors that reduce the training time required by users of invasive BCI systems (Schalk et al. (2008), supra, J. Neural Eng., 5:75-84). In addition, studies of tetraplegic humans with implanted microelectrode arrays have exclusively demonstrated 2D control of a cursor through imagined natural movement (Hochberg et al (2006), supra, Nature, 442:164-171; Kim et al (2008) “Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia,” J. Neural Eng., 5:455-476). This decoding of imagined natural movement is also a likely factor in reduced training time since neural signals directly correlate with intended actions.
Traditionally, the focus for many researchers of decoding movement intent from neural activity has been on decoding the desired hand position (Carmena et al. (2003) “Learning to control a brain-machine interface for reaching and grasping by primates,” PLoS Biol., 1, E42; Bradberry et al. (2010) “Reconstructing three-dimensional hand movements from noninvasive electrocephalographic signals,” J. Neurosci. 30:3432-3437; Georgopoulos et al. (2005), supra, Exp. Brain Res., 167:132-135; Hochberg et al (2006), supra, Nature, 442:164-171; Schalk et al. (2007), supra, J. Neural. Eng., 4:264-275; Velliste et al. (2008), supra, Nature, 453:1098-1101; Wessberg et al. (2000), supra, Nature, 404:361-365; Walden et al. (2008), supra, J. Neurosci., 28:1000-1008). The problem of decoding hand gestures, which involves simultaneously decoding multiple finger joint angles, is relatively complex due to the many degrees of freedom involved. Researchers have only recently started investigating the possibility of deciphering finger movement and simple open/close hand movements (Acharya et al. (2010) “Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand,” J. Neural Eng., 7(4):46002; Kubanek et al. (2009) “Decoding flexion of individual fingers using electrocorticographic signals in humans,” J. Neural Eng., 6(6): 66001). Moreover, the methods used in many prior studies have involved invasive procedures required to measure neural signals. Other studies have indicated that, in the case of noninvasive EEG signals, signal averaging must be used over many movement attempts to extract a usable signal and the extracted signal cannot be employed to reproduce a time-varying arm trajectory.
More recently, some advancements have been made in the development of wearable robots or “exoskeletons” for medical (e.g., restoration of walking and running after spinal cord injury, amyotrophic lateral sclerosis or traumatic brain injury to name a few), rehabilitation training (e.g., assistive or resistive movement therapies), and motor performance enhancement applications in which the exoskeletons are not being permanently integrated with the body. Such exoskeletons evolved directly from orthotic devices worn on the arm or leg to provide assistive movement for specific joints such as the elbow or knee.
Exoskeletons range from powered versions of these orthotic devices to full body suits designed for augmenting the strength and load-carrying capability of able individuals. In addition, exoskeletons are finding their way into rehabilitation therapy, as they can provide either assistance or resistance in patient conditions ranging from full incapacitation (e.g., immediate post-stroke or traumatic brain injury) to almost full-strength following weeks of therapy. While the evolution of exoskeleton hardware has proceeded at a rapid pace, controlling these devices has lagged significantly behind.
Most exoskeletons use force sensors mounted at the handle or along attach points on the limbs to generate commands to move the exoskeleton in the appropriate direction (Carignan et al. (2008) “Controlling shoulder impedance in a rehabilitation arm exoskeleton,” proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Pasadena, 2453-2458; Frisoli et al. (2008) “Robot mediated arm rehabilitation in virtual environments for chronic stroke patients: a clinical study,” Proc. IEEE Int. Conf. on Robotics and Animation, Pasadena, 2465-2470; Nef et al. (2007) “ARMin—exoskeleton for arm therapy in stroke patients,” Proc. IEEE Int. Conf. on Robotics and Animation, Noordwijk, The Netherlands, 68-74). However, such devices are not scalable as a human-machine solution for minimizing the time between thought and action as well as for patients with severe injuries who need some assistance just to lift or move their limbs.
Some researchers have investigated using (non-invasive) surface electromyography (sEMG) as an alternative way to decipher user intent (Rosen et al. (2005) “The human arm kinematics and dynamics during daily activity—toward a 7 DOF upper limb powered exoskeleton,” Int. Conf. on Advanced Robotics (ICAR), Seattle). In one strategy, noninvasive electrodes pick up signals from muscles that generate elbow flexion/extension, which are then processed by software to command movement of the elbow orthosis. The human then closes the loop through visual and proprioceptive feedback during the movement. Unfortunately, this strategy has fails to work for patients with severe motor impairment who do not generate enough neuromuscular activity to be captured by sEMG. Moreover, sEMG patterns are severely compromised by neurological disease, injury or noise during movement rendering them difficult to decode accurately.
Thus, brain computer interfaces (BCIs), or brain machine interfaces (BMIs) pose the best avenue to effective control of exoskeletons since it is a user's thought that commands motion. Bioengineers have had some success with using electrodes implanted in a monkey's brain to command motion of a robotic arm and gripper (Velliste et al. (2008), supra, Nature, 453:1098-1101). Bipedal locomotion control is of great interest to the field of BCIs. Since locomotion deficits are commonly associated with spinal cord injury (Scivoletto et al. (2008) “Prediction of walking recovery after spinal cord injury,” Brain Res. Bull., 78:43-51; Rossignol et al. (2007) “Spinal cord injury: time to move?,” J. Neurosci., 27:11782-11792) and neurodegenerative diseases (Boonstra et al. (2008) “Gait disorders and balance disturbances in Parkinson's disease: clinical update and pathophysiology,” Curr. Opin. Neurol., 21:461-471; Yogev-Seligmann et al. (2008) “The role of executive function and attention in gait,” Mov. Disord. 23:329-342, quiz 472), there is a need to investigate new potential therapies to restore gait control in such patients. While the feasibility of a BMI for upper limbs has been demonstrated in studies in monkeys (Carmena et al. (2003), supra, PLoS Biol., 1, E42; Velliste et al. (2008), supra, Nature, 453:1098-1101) and humans (Hochberg et al (2006), supra, Nature, 442:164-171; Bradberry et al. (2010), supra, J. Neurosci. 30:3432-3437), neural decoding of bipedal locomotion in humans has not yet been demonstrated. Recently, bipedal locomotion patterns were reconstructed from cortical ensemble activity in Rhesus monkeys recorded with implanted electrode arrays (Fitzsimmons et al. (2009) “Extracting kinematic parameters for monkey walking from cortical neuronal ensemble activity,” Front. Integr. Neurosic., 3:3 doi: 10.3389/neuro.07.003.2009). However, such invasive technology is not likely to be an acceptable solution for exoskeletons for use by humans.